{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Programming Exercise 1: Linear Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this exercise, you will implement linear regression and get to see it work on data. Before starting on this programming exercise, we strongly recommend watching the video lectures and completing the review questions for the associated topics."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get started with the exercise, you will need to download the starter code and unzip its contents to the directory where you wish to complete the exercise. If needed, use the cd command in Octave/MATLAB to change to this directory before starting this exercise."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also ﬁnd instructions for installing Python down below. These notebooks use Python 3.6 but should be compatible with Python 2 as well"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Operating System | Blog Post | Youtube Video\n",
    "--- | --- | ---\n",
    "Mac | [Install Anaconda on Mac](https://medium.com/@GalarnykMichael/install-python-on-mac-anaconda-ccd9f2014072#.lvhw2gt3k \"Install Anaconda on Mac\") | [Youtube Video](https://www.youtube.com/watch?v=B6d5LrA8bNE \"Youtube Video\")\n",
    "Ubuntu | [Install Anaconda on Ubuntu](https://medium.com/@GalarnykMichael/install-python-on-ubuntu-anaconda-65623042cb5a#.4kwsp0wjl) | [Youtube Video](https://www.youtube.com/watch?v=jo4RMiM-ihs)\n",
    "Windows | [Install Anaconda on Windows](https://medium.com/@GalarnykMichael/install-python-on-windows-anaconda-c63c7c3d1444#.66f7y3whf) | [Youtube Video](https://www.youtube.com/watch?v=KH2yIk03jFc&t=1s)\n",
    "Any | [Environment Management with Conda (Python 2 + 3, Configuring Jupyter Notebooks)](https://medium.com/@GalarnykMichael/install-python-on-windows-anaconda-c63c7c3d1444#.66f7y3whf) | [Youtube Video](https://www.youtube.com/watch?v=rFCBiP9Gkoo)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Files included in this exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ex1.m - Octave/MATLAB script that steps you through the exercise ex1\n",
    "<br>\n",
    "multi.m - Octave/MATLAB script for the later parts of the exercise\n",
    "<br>\n",
    "ex1data1.txt - Dataset for linear regression with one variable \n",
    "<br>\n",
    "ex1data2.txt - Dataset for linear regression with multiple variables \n",
    "<br>\n",
    "submit.m - Submission script that sends your solutions to our servers\n",
    "<br>\n",
    "[\\*] warmUpExercise.m - Simple example function in Octave/MATLAB \n",
    "<br>\n",
    "[\\*] plotData.m - Function to display the dataset \n",
    "<br>\n",
    "[\\*] computeCost.m - Function to compute the cost of linear regression \n",
    "<br>\n",
    "[\\*] gradientDescent.m - Function to run gradient descent \n",
    "<br>\n",
    "[†] computeCostMulti.m - Cost function for multiple variables \n",
    "<br>\n",
    "[†] gradientDescentMulti.m - Gradient descent for multiple variables \n",
    "<br>\n",
    "[†] featureNormalize.m - Function to normalize features \n",
    "<br>\n",
    "[†] normalEqn.m - Function to compute the normal equations\n",
    "<br>\n",
    "\\* indicates ﬁles you will need to complete \n",
    "<br>\n",
    "† indicates optional exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Throughout the exercise, you will be using the scripts ex1.m and ex1 multi.m. These scripts set up the dataset for the problems and make calls to functions that you will write. You do not need to modify either of them. You are only required to modify functions in other ﬁles, by following the instructions in this assignment. \n",
    "<br>\n",
    "   For this programming exercise, you are only required to complete the ﬁrst part of the exercise to implement linear regression with one variable. The second part of the exercise, which is optional, covers linear regression with multiple variables."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Where to get help"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just ask"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2 Linear Regression with One Variable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this part of this exercise, you will implement linear regression with one variable to predict proﬁts for a food truck. Suppose you are the CEO of a restaurant franchise and are considering diﬀerent cities for opening a new outlet. The chain already has trucks in various cities and you have data for proﬁts and populations from the cities.\n",
    "<br>\n",
    "You would like to use this data to help you select which city to expand to next.\n",
    "<br>\n",
    "The ﬁle <b>ex1data1.txt</b> contains the dataset for our linear regression problem. The ﬁrst column is the population of a city and the second column is the proﬁt of a food truck in that city. A negative value for proﬁt indicates a loss. The <b>ex1.py</b> script has already been set up to load this data for you."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1 Plotting the Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before starting on any task, it is often useful to understand the data by visualizing it. For this dataset, you can use a scatter plot to visualize the data, since it has only two properties to plot (proﬁt and population). (Many other problems that you will encounter in real life are multi-dimensional and can’t be plotted on a 2-d plot.) \n",
    "<br>In <b>ex1.py</b>, the dataset is loaded from the data ﬁle into the variables X and y:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np \n",
    "import matplotlib.pyplot as plt\n",
    "import plotData\n",
    "# from mpl_toolkits.mplot3d import Axes3D\n",
    "# from matplotlib import cm\n",
    "# import warmUpExercise as wue\n",
    "import computeCost as cc\n",
    "#import gradientDescent as gd\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data = np.loadtxt('ex1data1.txt', delimiter=\",\")\n",
    "X = data[:,0]\n",
    "y = data[:,1]\n",
    "m = len(y) # number of training examples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, the script calls the plotData function to create a scatter plot of the data. Your job is to complete <b>plotData.py</b> to draw the plot; modify the ﬁle and ﬁll in the following code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXu0JFV97z8/mDfCAM4EiYiIAZFLDI8ZROMYjojhMXdk\njAswccTEeOJJlBAgI5gcJMMyOCODUbkZooGASsQhASREFiJziCY3mnOG1wCjArkSMTCMooL4AvK7\nf+yq6To1Vd3Vfaq6q7u/n7X2Ot1Ve1f9urvO/u7Hb/+2uTtCCCGGl116bYAQQojeIiEQQoghR0Ig\nhBBDjoRACCGGHAmBEEIMORICIYQYciQEQggx5EgIhBBiyJEQCCHEkDOrqgub2UuATwP7AA580t0/\nZmYXAu8GtkdZP+DuX2x2rUWLFvkBBxxQlalCCDGQbN68+XvuvrhVvsqEAHgOOMfd7zSz3YHNZnZb\ndO6j7n5J0QsdcMABTE1NVWKkEEIMKmb2SJF8lQmBuz8GPBa9ftrMtgIvrup+QgghOqMrcwRmdgBw\nBPD16ND7zOxeM7vSzPbqhg1CCCGyqVwIzOwFwD8CZ7n7U8AG4EDgcEKPYX1OuVEzmzKzqe3bt2dl\nEUIIUQKVCoGZzSaIwDXufj2Au29z9+fd/X+ATwFHZ5V190+6+xJ3X7J4ccu5DiGE6H/WrYOJieZ5\nJiZCvhKpTAjMzIArgK3ufmni+L6JbCuB+6qyQQgh+oqlS+HUU/PFYGIinF+6tNTbVtkj+HVgFfAG\nM7s7SicB68xsi5ndC4wAf1KhDUII0T+MjMDGjdliEIvAxo0hX4lU6TX0r4BlnGq6ZkAIIYaapBjE\nlX6FIgDVriMQQgjRCUkxGBuDDRsqEwFQiAkhhKgnIyNBBC66KPytSARAQiCEEPVkYiL0BMbHw99W\n3kQzQEIghBB1IzknsGZN/gRySUgIhBCiTmRNDDfzJioBCYEQQtSFZt5BFYqBhEAIIerC5GRz76BY\nDCYnS72tuXupF6yCJUuWuMJQCzHErFsXVtM285yZmAgV5OrV3bOr5pjZZndf0iqfegRCiPrTo9AL\nw4KEQAhRf3oUemFYkBAIIfqDLDGQCJSCQkwIIfqHLodeGBbUIxBC9BddDL0wLEgIhBD9RRdDLwwL\nEgIhRP9QReiFHu0KVickBEKI/qCq0AtyTZUQCCH6gCpDL8g1VUIghOgDqg69MOSuqQoxIYQQMXHl\nPyCuqQoxIYQQ7RBPBjdzTR3QSWMtKBNCCAiTwStXgnvDNXVkpCEGyaGiAUNCIIQQMe5g1hCAZMU/\nwPMFEgIhhIhb+zfeGN7Hlf7GjXDKKUEcbrhhIEUAJARCiGEnyzsoGc/ILPQUBhhNFgshhpss19Rk\nPKMzzww9hZJ3BasTEgIhxHCzenW2d1AynlGcb0CREAghRJIq4hnVHAmBEELEVBXPqOZUJgRm9hIz\nmzCzB8zsfjP74+j43mZ2m5k9GP3dqyobhBCiMFXGM6o5VfYIngPOcfdDgWOAPzKzQ4HzgNvd/SDg\n9ui9EEL0lqrjGdWYrsUaMrMvAJdF6Vh3f8zM9gXucPdXNCurWENCCNE+tYo1ZGYHAEcAXwf2cffH\nolOPA/t0wwYhhBDZVC4EZvYC4B+Bs9z9qeQ5D92RzC6JmY2a2ZSZTW3fvr1qM4UQYmipVAjMbDZB\nBK5x9+ujw9uiISGiv09klXX3T7r7Endfsnjx4irNFEKIoaZKryEDrgC2uvuliVM3AWdEr88AvlCV\nDUIIIVpTZayhXwdWAVvM7O7o2AeADwMbzexdwCPAqRXaIIQQogWVCYG7/ytgOaePq+q+Qggh2kMr\ni4UQYsiREAghxJAjIRBCiCFHQiCEEEOOhEAIIYYcCYEQQpTBunWtI5NOTIR8NUNCIIQQZbB0afMw\n1XGY66VLu2tXASQEQghRBs32LGi210ENkBDk0cfdPCFEj8gSg5qLAEgI8unjbp4QoockxeCCC2ov\nAiAhyKePu3lCiB4zMgJjY3DRReFvzesJCUEz+rSbJ4ToMRMTsGEDjI+HvzXf51hC0Io+7OYJMXD0\n05xdsrG4Zk1fbHovIShCn3XzhBg4+mXOLmvEoNkwc02QEBShz7p5Qgwc/TBn18yOmouBhKAVfdjN\nE2Igqfuc3eRkczti+ycnu2tXASzsH19vlixZ4lNTU92/cd5DVqeHT4hhI/7/GxsLPXT9H+ZiZpvd\nfUmrfOoR5NHH3TwhBhrN2ZWOhCCPPu7mCTHQaM6udDQ0JIToH9I9dQ3TNqWyoSEz28vMXtWZWUII\n0SF96prZDxQSAjO7w8z2MLO9gTuBT5nZpdWaJoQQEZqzq5SiPYKF7v4U8Bbg0+7+auCN1ZklhBAJ\nNGdXKUWFYJaZ7QucCtxcoT1C9D/9FA6hX1i9uvUcwMhIyCfapqgQrAFuBR5y90kzOxB4sDqzhOhj\n+iUcghARhYTA3a9z91e5+x9G7//T3X+rWtOE6FP6IRyCEAlaCoGZ/aaZbTCzm6K0wcxO6IZxQvQt\ndQ+HIESCWc1OmtlfAQcDnwYejQ7vB5xpZie6+x9XbJ8Q/UtSDBQOQdSYpkIAnOTuB6cPmtnngW8B\nuUJgZlcCy4En3P2w6NiFwLuB7VG2D7j7FzuwW4j+IBkOYXxcIiBqSauhoZ+ZWdaM1lLgZy3KXgVk\nDSF91N0Pj5JEQNSLsj1+FA5B9AGthOCdwGVm9oCZfSlKW4GPR+dycfevAE+WYqUQ3aJMjx+FMBd9\nQlMhcPc7o8VjbwDOj9KIux/j7ps7vOf7zOxeM7vSzPbq8BpCVENZHj8KhyD6iCJeQwa8NJmiY52w\nATgQOBx4DFjf5L6jZjZlZlPbt2/PyyZE+czU40fhEESf0VQIzOxNhIVjFwInRekvgAejc23h7tvc\n/Xl3/x/gU8DRTfJ+0t2XuPuSxYsXt3srIWZGssK+4IL23D4VDkH0Ga28hj4GvNHdv508aGYvA74I\nvLKdm5nZvu7+WPR2JXBfO+WF6CqdevwUCXMwMiIPIlEbWg0NzaKxfiDJd4HZzQqa2eeAfwdeYWaP\nmtm7gHVmtsXM7gVGgD/pwGYhuoM8fsSQ0KpHcCUwaWbXAt+Jjr0EOB24ollBd39bxuGmZUSHrFsX\nvFiatTAnJsJQhIJyFSM9zj8yolXBYmBp5TV0MfA7gAGviZIBvxOdE3VAQc7KRR4/Ysho6TXk7g+4\n+4fd/X1R+rC7P9AN40RBFOSsPZotGou/r/PP33kyV2IgBpRWXkMLzezDZvYNM3vSzL5vZlujY3t2\ny0hRAAU5K06zHtTkZBCBiy/O7kHJ40cMIE03rzezW4FNwNXu/nh07EXAGcBx7t62C2knaPP6Nogr\nfwU5a06eSEo8xQBR1ub1B7j72lgEANz9cXdfS1hcJupG0uVxbEyVWR7qQQmxg1ZC8IiZrTazfeID\nZraPmb2fhheRqBNyeSzOTBaNCTFAtBKC04AXAv8SzRE8CdwB7E3Yv1jUCQU5ax/1oIRo6T76A3d/\nv7sf4u57R+mV0TFFFq0TcnnsDPWghCi8ef00zOzNZvbqso0RHaIgZ52hHpQQQIdCALwa+HMzu6VM\nY0SHKMhZ+6gHJcQOmrqP1gW5j4pSaeUdJO8hMSCU5T4aLyo7zczOjtJpWkw2AJS9JWM/Ucce1DD/\nHqLntFpZ/A7gTuBYYEGURoDN0TlRF9qtSIY5PtHq1a1b+iMj3Q3QN8y/h+g97p6bgG8Ce2Yc3wv4\nVrOyZaajjjrK+461a903bWqeZ9OmkK+Me2za5L5o0c73jO+Rdb5Zmazjolr0e4iSAaa8QB3bSgi+\nBSzMOL4QeLDIDcpIfSkErf55y/jnTl8j7/369fn3anUN0V30e4gSKUsIzgAeJuw1/IEoXR4de2eR\nG5SR+lII3LvTwmtV+TcTgfQ1xsdV6dQB/R6iJEoRgnAd9iJsRHNOlE4H9ipy8bJS3wqBe3daeOlr\nrl/vbua+alXxe42Ph8dhfLw8u0Tn6PcQJVCaENQh9bUQuHenhZe+x6pVxSsStUDrhX4PURKVCwGw\npdOy7aa+FwL37rTw4nvEPYEiFYnGpOuFfg9RImXNEbwlJ/0WsL3IDcpIfS8E3ewRrFoVhoXWr59+\nvMhEcavjolr0e4iSKUsIngWuAv4uIz1d5AZlpL4Wgm7OEeRNELfjOlr0vCgX/R6iAsoSgs3AYTnn\nvlPkBmWkvhWCbnoNtar80++7sc5BFEe/h6iAsoRgGbB/zrklRW5QRupLIej2OoKsiiRLDFSRCDE0\nFBUCBZ2rinXrQjiAZqEMJiZCPJtOQxl04x5CiL6laNC5lkJgZr8EPOPuz5jZfOBsYHfgY+7+WCnW\ntqAvhUAIIXpMadFHgWsJ21UC/AXwK8APgL/v3DwhhBB1oVX00TOAlwPHRq9PA6aAx4GXmtk7zOxV\n1ZspRMko7LMQO2jVI7gDeAa4F3gU2Ab8U3T8e9HfRyqzToiqUNhnIXbQavP6R4BPALcCnwXWuPt/\nAQ58393/y91/lFXWzK40syfM7L7Esb3N7DYzezD6u1d5H0WINmi2LaV2KBNDRss5AnffQBgeerm7\n3xwd/j7wthZFrwJOSB07D7jd3Q8Cbo/eC9EbssRAIiCGkEKb17v7j939J4n3z7j7D1uU+QrwZOrw\nm4Gro9dXA6e0YasQ5ZMUgwsukAiIoaSQEJTIPgmX08eBffIymtmomU2Z2dT27du7Y50YTkZGYGwM\nLroo/JUIiCGj20Kwg2jVW+4iBnf/pLsvcfclixcv7qJlYuiYmIANG2B8PPxt5U0kxIDRbSHYZmb7\nAkR/n+jy/eU2KKaTnBNYs2b6nIGeFTEkFBICM3tL5OnzIzN7ysyeNrOnOrjfTYTtL4n+fqGDa8wM\nuQ2KmKyJ4eScwaxZelbEcFAkIBHwEPDKInkTZT4HPEYIZf0o8C7CCuXbgQeBLwN7F7lW6UHnFPdd\nFA0KmLfns54V0QdQMOhc0aGhbe6+tU2BeZu77+vus919P3e/wt2/7+7HuftB7v5Gd097FXUHuQ2K\nycnmv3X8jDz3nJ4VMfAUij5qZh8DXgTcCPw8Pu7u11dnWoPKgs7F/9BjY2GSUP/YIg89K6IPKRp0\nblbB6+0B/AR4U+KYA10RgspIug2Oj+sfW+SjZ0UMMIWEwN1/t2pDekLabXBkRP/gIhs9K2KAaSoE\nZrba3deZ2SfI8Pl39zMrs6xq0uO8IyMa9xXZ6FkRA06ryeJ4gniKsH9xOvUnrdwGtaCoXvTSn1/P\nihgGirgW9TqV6j7ajb2ERbn06jfTsyL6HEp2Hx0ciroNTk52165+pFst9V6FjNazIoYEbV4vOqdV\nJVx2JZ2+nvz5hWhKmXsWY2a/XuSYGDBatfhHRuD882H58u601BUyWohKKDo09ImCx8QgUSQu08UX\nB9/6bq28VchoIUqn1eb1rzGzc4DFZnZ2Il0I7NoVC7tFP0aarNrmomPzZ5/dvZa6QkYLUTqtegRz\ngBcQ1hvsnkhPAW+t1rQu049RSbthc9G4TEVb6jMRr2Yho4UQnVPEtQh4aZF8VaXSo4/m0Y9RSbtl\nc3y98fHm98s7X9Sudj9PnX8bIXoMBd1HWwnAX0V//4mwl8C0VOQGZaSuCMHataEySVcsyfebNoV8\nVdy3Ga3u28zmMu85Ph4emfHx6ddKh2rOsid5rbIre4mBEJmUJQRHRn9/IysVuUEZqStCkK7w063b\nbi5aSlbUzSrHrMq1VYu82TWbnU9ff3Q0HFu/3t0s/E1fY4893JcvL1a5N7OpDLEUYggpSwhuj/6u\nLXKxqlJPhoaSrd90JVV2xZRXKWZtipLVAo+JbV61qvW922l9Z9m3cKH7bruFyj7Lnk2bwvkskUhf\nt4h4CSHapiwheAB4LSHm0BHAkclU5AZlpK4JgXujJbtwYaigFi4M75tVjFnXaLdiS5fJa2k3a4Ev\nWhREoFnl2+yeRUQgeXzevFDZp3tMye9w/frmopQ13CSEKIWyhOCtwC3A08BEKm0qcoMyUteFIK78\nx8cbFVo3Ji/TLeS8sfdWx/N6DEXu2a7gJYUyfj9/fvZ31s69hRAzphQh2JEJxovkqyrVbmgoK3/W\n+05It5DzKstWlX87tuS1yosOgY2ONmycP79YC7+K704IMY1ShSBcjxXAJVFaXrRcGam2k8XJ8fpm\nbpXtzhWkr5VXUcfDRKtWdd47KatVHtu4YEHnLqQSAyFKpewewcXA7cDvRek24C+LlC0j1dZ9NDku\nn1VRt1OxtZowzqpc167Nv3fyunlCVFarPJ4TWLBg+jCRXEGF6CllC8G9wC6J97sC9xYpW0YqLAQz\n9ebppPLKa5XPRATS144nfuN8setmq/H9dtYeFPmseddJzqNk9azS4jWT30gIUZgqhGDvxPu9aykE\nM2ltdlI2b5y+k8navAo5a+x/jz0arpt5PZeZtLqLikFaBLLKq4UvRM8oWwjeBjwCXAVcDfw/4LQi\nZctIbQ0NddrSLdJSHR0NKet68fvjjw9fa+w22c4182xNt5DXrw/3iF03k+XyvJxafdb0saxWefJY\nuneSptmQWieoJyFE25QmBIABLwH2jSaMVwAvKnLxslLbcwRljX2nGR1t+MYvWLCzr348Xn/QQeF+\nY2MhX7st72aVXrKnEHvrJD9n1rqHIrTbS+h2xay5BSHapuwewZYi+apKbQlBetK3k/HzvOuuXx8q\n2XnzQiWfXgAG7rvsEirjsbHQYs8bJuq04spquWd5OJ14YvsVdVnzBlVRd/uEqBllC8HVwNIieatI\nHQ8NZblddlppJFviyVWzccsffMeQUN4kb14PZaat604+Z9FKtW6VbN3tE6JGlC0E3wCeBx6OJo63\n1HKyOCY5Vl5m4LgsMYjnBGIRyJvkHR1trFZuNumcVdk3s7tVz2cmLpx1XfFbd/uEqAllC8FLs1KR\nsjnX+3YkJncXMbQjIcgKfdDJ2HnWteNKe86chgjstlv+6t/YhgULdm65x7SKIdRqQjYvb6ct6LrH\nAKq7fULUgLJiDc0DzgIuA/4AmFXkoi1vGoRgUdH8HQ8NJVuOaW+amQzHxD2OXXYJX+Hcuc0XlcX3\nSi+6StvcTpiIdlr77bag697irrt9QtSEsoTg88BnIxG4EfhYkYu2vGlVQpBVCebFDJqJF8qmTaHy\nB/dly0LlXjTUQ1ZvJavyHx9vTDY3sy1LrFp9D+18h3Ubg6+7fULUiLKEYEvi9SzgziIXbXnTsA7h\nbmAzMJqTZxSYAqb233//Yp+6qEdN2he+nTH0TZtCBR3H/Y8Xdy1YMH0CudVwzcKF7rNn77wWwH36\nvgILF+68ziAvHEbazvTn7PcYQHW3T4iaUZYQ3NnsfacJeHH095eAe4DXN8vfUayhrJZjVgs7nW/9\n+uzKOXmN5MRw3LrP8iZqtsL4uON8x/zCySfn9whmz86O65+0u+i6gzjfTCaQe1XZ1t0+IWpIWULw\nPPBUlJ4Gnku8fqrIDVoaABcC5zbLM+MFZTEz3dTl5JOni0B6mCkWg3i1bbOhnThu/7x50yv7+JrJ\n1cOxe+rYWPbnS1f26c9VpJdS95W7dbdPiBpSqtdQmQnYDdg98fr/Aic0K1NKiImYPDGIJ3tXrcq/\nbrJizxuGOvnk6WEo0hXT6Gi4TiwW8dDS3LkNu+KKP7kmYcWK5vMQeWsXmi1oyxOqdD5VrkL0JXUW\nggOj4aB7gPuBP2tVpi0hOPHE1pXb2FhohSeHg5pVsu7tb9LSbGgpWWGPjjYmnufODXbFrf+4Mo9D\nVcRrFtJilVXpN9vbOCYOlaHhFiEGktoKQSep1B5ButJMDwe1Kl9EELLmGdLzCclhpXnzggjE7qiz\nZ++8XWUsBq3sLTox3Or7kggI0fcMrxC471yJ5XnZJIeDingTtTqXPJ+3sjhrnH7TpkZPYJ99Gts9\nxj2U9LVa9WDaXWyVJyYSASH6muEWAvedK9qk7/7ataGFHVemWauOW3kPtROnp9k2losWNYZ8Zs8O\nrf7ddgurls12Dm4Xkzen0W6PYKblhBC1RULgPr1yS64sjj1yjjlmuttnlhfO6Gix+D954hC3zo87\nLrtyjUXg+OMbk8Rz5uw8gZx176LDREUrdYVtEGKgkBDEpFcWJ8NIx2sCkhOzyTUAo6OhhT421lwM\nVq3KnnTNE6KYuOI//vjG6uRYGObPd1++vLFQLa9yj3suMw13rR6BEAOHhMB958pt/fpQqS5Y0Ejz\n5vlOrppjY43hongIKc8TqdUQTbICz+qVjI017jt3biNianJiOfZEyqvs81xi82xpdV5zBEIMBMMr\nBHkTw5s2NXYViyvv+H3cM4gXb82bN927p1UFXHTj+uS6gXnzGgvT4l5HVsTUeE1CM1fPIq6ieesB\n5DUkxMAyvEKQnhhOn0sGiItb6LEwxL78aRfPuGyW507RBVwxyRXDaRfRtIAlX1exsrbdnoIQoq8Y\nbiHI28A9rrzjlnVSGJYta4jBsmXNW8l5oSiSvY484tZ7HGsoywW0qCvrTFHYBiEGmuEVgryhoWQL\nPtnCnjfPfdYs3+GtE79OjvmPjjaGaJrNCTQL6pbOG89dtBKO5LWFEKINhlcIkjRrwcfnkvsNz5sX\nKuY45EMsGsmoorvt1npOoFnFrYlZIUSXkBDEJGP7xyTH82MvonQU0OQEcuzl08mcQBJNzAohushw\nC0F6eCjtGx+vLE56B6X3Ehgba+xJHK/uzQv10Gpox10Ts0KIrjPcQpAczonDPScr2rgCX7585z0E\n0kNJyZDQ8bXTaxOKVOCamBVCdJmiQmAhb71ZsmSJT01NFS8wMQErV4aR/9e/Hr76VbjhhnBu5Ur4\n+c/hQx+C554L5770JTjwQPjrvw5lL7oIVq2Cww6Dhx+G734XXvziUP7lL4cf/zjkGR+HNWtCmclJ\nWL26/A8vhBAdYmab3X1Jq3y7dMOYrjIxAaeeGir+G28MFf2zzwYBuOuuIA5z58IRR8CsWfDP/wy7\n7ALf+Q4sXw4f/3io4G+5JZy//np4wxvg85+Ha68NxzZsCHk2bAj3GxmRCAgh+pZZvTagdCYnYePG\nUDlDEISVK+GnP4Vzz4VLLgkicO21oXKfPx9mz4bf/m24/PJQZmQE9twz5H/Pe0Kr3x0++EG4+OLG\n9UdGgugk7yeEEH3GYA4NpbnggjCUM2dOqPjPPDO0/N1Dr+GuuxqV/t//fehBJMVh/vzQK4iHmJKV\nftwD2bgxvNcQkRCiJgzv0FCaiYnGUM78+aFncNFF8ItfBBGA0Mq/5BK47jpYsQJ+8hN45hm4+upQ\nxixcJymaExOwbl0QhY0bQ6/jlFNg6dLefE4hhOiQwRsaSpJsrcfDPeecE86Zhb/JoaQf/jCIxK/+\nKmzZEiaTx8dh27aQ9/TTw/XOP78xRBTjHs6nh4jWrQvi0GzoSJPNQoheUsS1qNdpxjuUxe8XLgz7\nAsc7gWXF+T/ySN+x0jiOUJp0+4yDxp18cvZ9WtnR7nkhhOgQhn4dQZYIxPsLJMNJJBeRrV/f2Dt4\nzpzGXgTJIHVxGOk99tAG8UKIWjPcQpBcvJUVljpu1b/ylY3oo/FKY3BfsaJR4R9zTDg/Z04QiTjk\nRN62jkXj/ksEhBAVU1QIBn+y+NprwyDPIYcE7yCAs88Ok8Jbt4Z5gLlz4W/+Bn72M9h/f/jCF4Kr\n6DPPhLmCN74xTC4/+2wof9ddO68lgMacRNaEcTypfOqpwYtJbqdCiLpQRC16nWY0NBT3DtJhqBct\nCj0CcN99d98xJ5A8Hwef23XXEJ569uxGD6HT4HPaIF4I0SUY6qEh9+yhl/TGNO7uL3tZtgjEcwJx\n4LlVq6bvaRxvGJ+8btZGNclhorwgeEIIUQESgqx9fDdtakwGJwPKxemooxqt+9HR6dtaJjeTj6OT\nJvcrbrW1ZZlzBApgJ4QogIQgPWSTjBYa7zWQ7AkcdVR4ffDBodKfO3d6K390tOFuGl/7+OMbvYX4\nnlm7lJXtNSSXVCFEASQE7tPDSifH5VesaIjAsmWNfAcfHI7tsUf+UE8ylHW8W1nWnEHRln/ZYiAR\nEEJE1FoIgBOAbwIPAee1yj+jHcri8ft4CCcpAvH+xPG+wYsWue+3X6NnkEWeO2ryHslKuMphHLmk\nCiGaUFshAHYFHgYOBOYA9wCHNisz4x5BXMnHPYN4OCheTxBvSbl+fRCFuGeQtfNYvKgs3sw+Jm9T\n+6rRBLQQIoc6C8FrgFsT788Hzm9WphSvodhtM90DSIpDvFjMPdsTqNVwTHqYqFvIJVUIkUGdheCt\nwN8m3q8CLsvINwpMAVP7779/e58+a8gkuRl9PL4f70Wc7inExPmSex23GpOPexXdapmrRyCEyKHv\nhSCZOgoxkd6r+Oijp68TiENKLF8+3Zvo6KOnX6vZuHuvJ2w1RyCEaEKdhaB7Q0PxBvbpijG5sCwW\njLhMXsWenszttQtnr0VICFF7igpBL2INTQIHmdnLzGwOcDpwUyV3Mpu+mQyEeEBr1jS2qEzuIZCM\nBxTHD4qPp/cKSG+JmSa+1uRkeZ8n+RnyYhXlfQYhhMih6xvTuPtzZvZe4FaCB9GV7n5/6TeanITT\nToNXvGJ6pRkHobv55pDv2munbwqTrMCbBYQrsolMvK9x2bQjQgpqJ4RowWDvWRy3nOMdxcbGQrTQ\nuBJt1rIWQog+R3sWQ6NlfPHFcOKJYRvKsbHORWDdutbDLfFexkII0ScMthBAqOTPPx8++1lYtSr0\nCC69tLOewNKlzcfem+1HIIQQNWXwhWBiIvQILrkEbrkl9AzOPTeIQ7vDQc0mYjXMJIToUwZbCJKV\n89lnh2Ghz3wG3v72IA6deNVkiYFEQAjRxwyuEKQr54mJxvaSt9wSegSdulhq20khxADRdffRrpF0\nsUyLwshIw5uoUxfLkZHQw7jooiAuEgEhRJ8yuEIQ+/lnDdskW/QbN3Z2/WQPY8OG6tYMCCFExQzu\n0BBUtwI3ed01a7SSVwjR1wy2EFQRBqJVD0NiIIToMwZrZfG6dcGHv9kQzcTE9JAS7dDKO0jeQ0KI\nGjGcK4uvo4hhAAAKdklEQVSrXvDVy0BzQghREYPVI4D8Vrla60KIIWM4ewSgBV9CCNEmg+k+mhSD\ndMRRIYQQ0xi8HkFMcsFXHHFUCCHETgyuEKQXfMmtUwghMhlMIdCCLyGEKMzgCYEWfAkhRFsMlhBo\nU3chhGibwRICLfgSQoi2GbwFZUIIIYBhXlAmhBCiLSQEQggx5EgIhBBiyOmLOQIz2w480mHxRcD3\nSjSnamRv9fSbzbK3WvrNXihu80vdfXGrTH0hBDPBzKaKTJbUBdlbPf1ms+ytln6zF8q3WUNDQggx\n5EgIhBBiyBkGIfhkrw1oE9lbPf1ms+ytln6zF0q2eeDnCIQQQjRnGHoEQgghmjAwQmBm3zazLWZ2\nt5ntFI/CAh83s4fM7F4zO7IXdka2vCKyM05PmdlZqTzHmtmPEnku6LKNV5rZE2Z2X+LY3mZ2m5k9\nGP3dK6fsCWb2zei7Pq/HNn/EzL4R/eY3mNmeOWWbPj9dtPdCM/tu4nc/Kads17/jHHs/n7D122Z2\nd07ZXny/LzGzCTN7wMzuN7M/jo7X8jluYm/1z7C7D0QCvg0sanL+JOAWwIBjgK/32ubIrl2Bxwn+\nvsnjxwI399Cu1wNHAvcljq0Dzotenweszfk8DwMHAnOAe4BDe2jzm4BZ0eu1WTYXeX66aO+FwLkF\nnpmuf8dZ9qbOrwcuqNH3uy9wZPR6d+BbwKF1fY6b2Fv5MzwwPYICvBn4tAe+BuxpZvv22ijgOOBh\nd+90wVwluPtXgCdTh98MXB29vho4JaPo0cBD7v6f7v4L4NqoXOVk2ezuX3L356K3XwP264YtRcj5\njovQk++4mb1mZsCpwOeqtqMo7v6Yu98ZvX4a2Aq8mJo+x3n2duMZHiQhcODLZrbZzEYzzr8Y+E7i\n/aPRsV5zOvn/PK+NuoO3mNn/6qZROezj7o9Frx8H9snIU9fvGeD3CL3CLFo9P93kfdHvfmXOsEUd\nv+NlwDZ3fzDnfE+/XzM7ADgC+Dp98Byn7E1SyTM8SELwOnc/HDgR+CMze32vDWqFmc0BVgDXZZy+\nE9jf3V8FfAK4sZu2tcJDX7RvXM7M7M+A54BrcrLU5fnZQBiOOBx4jDDc0g+8jea9gZ59v2b2AuAf\ngbPc/ankuTo+x3n2VvkMD4wQuPt3o79PADcQunZJvgu8JPF+v+hYLzkRuNPdt6VPuPtT7v7j6PUX\ngdlmtqjbBqbYFg+nRX+fyMhTu+/ZzN4JLAd+J/rH34kCz09XcPdt7v68u/8P8KkcO2r1HZvZLOAt\nwOfz8vTq+zWz2YRK9Rp3vz46XNvnOMfeyp/hgRACM9vNzHaPXxMmV+5LZbsJeIcFjgF+lOge9orc\nVpSZvSgad8XMjib8Vt/vom1Z3AScEb0+A/hCRp5J4CAze1nU4zk9KtcTzOwEYDWwwt1/kpOnyPPT\nFVLzVitz7KjVdwy8EfiGuz+adbJX32/0/3MFsNXdL02cquVznGdvV57hKmfBu5UIXel7onQ/8GfR\n8fcA74leG/B/CJ4AW4AlPbZ5N0LFvjBxLGnve6PPcg9hgui1Xbbvc4ShiWcJ46PvAl4I3A48CHwZ\n2DvK+8vAFxNlTyJ4PDwc/xY9tPkhwljv3VG6PG1z3vPTI3s/Ez2f9xIqnn3r8h1n2Rsdvyp+bhN5\n6/D9vo4w7HNv4vc/qa7PcRN7K3+GtbJYCCGGnIEYGhJCCNE5EgIhhBhyJARCCDHkSAiEEGLIkRAI\nIcSQIyEQLTGz56OIhveZ2XVmtqDk67/TzC5rkedYM3tt4v17zOwdZdqRcc+PRFEgP5Jx7kQzm4oi\nRd5lZuvTdkWf65fbvOffmtmhbeQ/xMz+3cx+bmbnps61jJ5pZnMtRBB9yMy+HoU2iM+dYSFC54Nm\ndkbi+MuivA9FZee08xlF/ZAQiCL81N0Pd/fDgF8Q1jt0m2OBHULg7pe7+6crvuco8Cp3/9PkQTM7\nDLgMeLu7HwosIfh6p+16J8HXuzDu/vvu/kAbRZ4EzgQuSdm4K2HdzImECJZvyxGYdwE/cPdfAT5K\niG6Jme0NfBB4NWGF6gcTcY/WAh+NyvwguoboYyQEol2+CvwKgJmdHfUS7rNoPwUzO8BC7PRrzGyr\nmf1D3IOwEC99UfR6iZndkb64mf3vqLV5l5l92cz2iVqp7wH+JOqZLLMQt//cqMzhZvY1a8Rr3ys6\nfoeZrTWz/zCzb5nZsoz7WdTyv89CLPfTouM3AS8ANsfHEqwGPuTu3wDwEBJiQ1TuQjM718zeShCI\nayKbTzazGxP3Pd7Mbsiw5w4zWxK9/rGZfcjM7ok+307B0dz9CXefJCzySlI0emYyEuc/AMdFK1x/\nE7jN3Z909x8AtwEnROfeEOWFRPROM/sNa+xNcFe80lXUHwmBKIyFmDInAlvM7CjgdwktxmOAd5vZ\nEVHWVwB/7e6vBJ4C/rCN2/wrcIy7H0GovFa7+7eBywmt0MPd/aupMp8G3u8hQN8WQks2Zpa7Hw2c\nlToe8xZCgLdfI4RK+IiZ7evuK2j0hNIxdA4DNjf7EO7+D8AUITbM4cAXgUPMbHGU5XeBK5tdg7D6\n/Gvu/mvAV4B3t8ifpGj0zB35PIQ6/hFh5W1e+RcCP/RGWOTkdc8F/ij6vMuAn7Zhr+ghEgJRhPkW\ndp6aAv6LEA/ldcAN7v6Mh+B41xP++QG+4+7/Fr3+bJS3KPsBt5rZFuBPgabht81sIbCnu/9LdOhq\nwgYqMXHgrs3AARmXeB3wuahVvw34F2BpG/YWwsMS/s8Ab7eww9RryA8nHPML4ObodZ79deLfgEvN\n7EzCb/JcqwKiHkgIRBHilvHh7v6+aKihGem4JfH752g8c/Nyyn4CuMzdfxX4gyb5ivLz6O/zwKwZ\nXivmfuCoDsr9HfB2QrDB6wpUlM96IwZMu/YXjZ65I1/U41tIiIGVV/77hE2dZqWO4+4fBn4fmA/8\nm5kd0oa9oodICESnfBU4xcwWWIh2uDI6BrC/mb0mev3bhOEeCFvpxRXob+VcdyGNCuuMxPGnCdv3\nTcPdfwT8IDH+v4rQqm/nc5xmZrtGwzavB/6jRZmPAB8ws4MBzGwXM8uaQJ9ms7v/N/DfwJ8TRKFK\ncqNnmtl7zey9Ub5kJM63Apsi8bkVeJOZ7RXNubwJuDU6NxHlhUT0TjN7ubtvcfe10f0lBH2ChEB0\nhIct9a4iVJpfB/7W3e+KTn+TsDHGVmAvwmYrAH8BfMzCxtrP51z6QuA6M9sMfC9x/J+AlfFkcarM\nGYSx/XsJ4/1r2vgoNxCiPd4DbCLMSTzerIC730uYc/hc9BnvI0R/THMVcHlk8/zo2DWEobOtbdiY\ni4Vw5Y8CZwN/bmaPmtkeUW/jvYQKfSuw0d3vj4odQiOk+RXAC83soega50Wf8UngIkKFPgmsiY4B\nvB84OyrzwugaAGdFk+73EiavWw19iZqg6KOiVCIPn5sjV1ORwsJ6ibvc/YqWmauz4WbgLQWG+MSQ\nUNaYqRCiBVEv5xngnF7a4e7Le3l/UT/UIxBCiCFHcwRCCDHkSAiEEGLIkRAIIcSQIyEQQoghR0Ig\nhBBDjoRACCGGnP8PLcEnGcKNAYgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x199604f7588>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(X,y,'rx', markersize=10, label='Training data') \n",
    "plt.ylabel('Profit in $10,000s')\n",
    "plt.xlabel('Population of City in 10,000s') \n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 Gradient Descent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this part, you will ﬁt the linear regression parameters θ to our dataset using gradient descent.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.1 Update Equations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<img src=\"images/costfunctionGradientDescent.png\" alt=\"Gradient Descent Cost Function\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 2.2.2 Implementation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In <b>ex1.py</b>, we have already set up the data for linear regression. In the following lines, we add another dimension to our data to accommodate the θ<sub>0</sub> intercept term. We also initialize the initial parameters to 0 and the learning rate alpha to 0.01."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_padded = np.column_stack((np.ones((m,1)), X)) # Add a column of ones to x\n",
    "theta = np.zeros((2, 1)) # initialize fitting parameters\n",
    "\n",
    "# Some gradient descent settings\n",
    "iterations = 1500\n",
    "alpha = 0.01"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.3 Computing the Cost J(θ)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you perform gradient descent to learn minimize the cost function J(θ), it is helpful to monitor the convergence by computing the cost. In this section, you will implement a function to calculate J(θ) so you can check the convergence of your gradient descent implementation. \n",
    "<br>\n",
    "Your next task is to complete the code in the ﬁle <b>computeCost.py</b>, which is a function that computes J(θ). As you are doing this, remember that the variables X and y are not scalar values, but matrices whose rows represent the examples from the training set. \n",
    "<br>\n",
    "Once you have completed the function, the next step in ex1.m will run computeCost once using θ initialized to zeros, and you will see the cost printed to the screen. You should expect to see a cost of 32.07"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.4 Gradient Descent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, you will implement gradient descent in the ﬁle gradientDescent.m. The loop structure has been written for you, and you only need to supply the updates to θ within each iteration. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you program, make sure you understand what you are trying to optimize and what is being updated. Keep in mind that the cost J(θ) is parameterized by the vector θ, not X and y. That is, we minimize the value of J(θ) by changing the values of the vector θ, not by changing X or y. Refer to the equations in this handout and to the video lectures if you are uncertain. \n",
    "<br>\n",
    "A good way to verify that gradient descent is working correctly is to look at the value of J(θ) and check that it is decreasing with each step. The starter code for <b>gradientDescent.py</b> calls computeCost on every iteration and prints the cost. Assuming you have implemented gradient descent and computeCost correctly, your value of J(θ) should never increase, and should converge to a steady value by the end of the algorithm. \n",
    "<br>\n",
    "After you are ﬁnished, <b>ex1.py</b> will use your ﬁnal parameters to plot the linear ﬁt. The result should look something like Figure 2: Your ﬁnal values for θ will also be used to make predictions on proﬁts in areas of 35,000 and 70,000 people. Note the way that the following lines in ex1.m uses matrix multiplication, rather than explicit summation or looping, to calculate the predictions. This is an example of code vectorization in Python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "predict1 = np.array([1, 3.5]).dot(theta)\n",
    "predict2 = np.array([1, 7]).dot(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.4 Visualizing J(θ)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To understand the cost function J(θ) better, you will now plot the cost over a 2-dimensional grid of θ<sub>0</sub> and θ<sub>1</sub> values. You will not need to code anything new for this part, but you should understand how the code you have written already is creating these images."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the next step of <b>ex1.py</b>, there is code set up to calculate J(θ) over a grid of values using the computeCost function that you wrote."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Grid over which we will calculate J\n",
    "theta0_vals = np.linspace(-10, 10, 100)\n",
    "theta1_vals = np.linspace(-1, 4, 100)\n",
    "\n",
    "# initialize J_vals to a matrix of 0's\n",
    "J_vals = np.zeros((len(theta0_vals), len(theta1_vals)))\n",
    "\n",
    "# Fill out J_vals\n",
    "for i in range(len(theta0_vals)):\n",
    "    for j in range(len(theta1_vals)):\n",
    "        t = [[theta0_vals[i]], [theta1_vals[j]]]\n",
    "        J_vals[i,j] = cc.computeCost(X_padded, y, t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After these lines are executed, you will have a 2-D array of J(θ) values. The script <b>ex1.py</b> will then use these values to produce surface and contour plots of J(θ) using the surf and contour commands. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Figure 3: Cost Function J(θ)</b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "Unknown projection '3d'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "\u001b[1;32mC:\\Users\\mgalarnyk\\AppData\\Local\\Continuum\\Anaconda2\\envs\\py36\\lib\\site-packages\\matplotlib\\projections\\__init__.py\u001b[0m in \u001b[0;36mget_projection_class\u001b[1;34m(projection)\u001b[0m\n\u001b[0;32m     64\u001b[0m     \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 65\u001b[1;33m         \u001b[1;32mreturn\u001b[0m \u001b[0mprojection_registry\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget_projection_class\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mprojection\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     66\u001b[0m     \u001b[1;32mexcept\u001b[0m \u001b[0mKeyError\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mC:\\Users\\mgalarnyk\\AppData\\Local\\Continuum\\Anaconda2\\envs\\py36\\lib\\site-packages\\matplotlib\\projections\\__init__.py\u001b[0m in \u001b[0;36mget_projection_class\u001b[1;34m(self, name)\u001b[0m\n\u001b[0;32m     28\u001b[0m         \"\"\"\n\u001b[1;32m---> 29\u001b[1;33m         \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_all_projection_types\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mname\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     30\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mKeyError\u001b[0m: '3d'",
      "\nDuring handling of the above exception, another exception occurred:\n",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-13-7ba3c4fb252b>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[0;32m      2\u001b[0m \u001b[0mfig\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mplt\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfigure\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      3\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 4\u001b[1;33m \u001b[0max\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mfig\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mgca\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mprojection\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'3d'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      5\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      6\u001b[0m \u001b[0mtheta0_vals\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtheta1_vals\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmeshgrid\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtheta0_vals\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtheta1_vals\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;31m# necessary for 3D graph\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mC:\\Users\\mgalarnyk\\AppData\\Local\\Continuum\\Anaconda2\\envs\\py36\\lib\\site-packages\\matplotlib\\figure.py\u001b[0m in \u001b[0;36mgca\u001b[1;34m(self, **kwargs)\u001b[0m\n\u001b[0;32m   1367\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1368\u001b[0m         \u001b[1;31m# no axes found, so create one which spans the figure\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1369\u001b[1;33m         \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0madd_subplot\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1370\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1371\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0msca\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0ma\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mC:\\Users\\mgalarnyk\\AppData\\Local\\Continuum\\Anaconda2\\envs\\py36\\lib\\site-packages\\matplotlib\\figure.py\u001b[0m in \u001b[0;36madd_subplot\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m   1001\u001b[0m         \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1002\u001b[0m             projection_class, kwargs, key = process_projection_requirements(\n\u001b[1;32m-> 1003\u001b[1;33m                 self, *args, **kwargs)\n\u001b[0m\u001b[0;32m   1004\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1005\u001b[0m             \u001b[1;31m# try to find the axes with this key in the stack\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mC:\\Users\\mgalarnyk\\AppData\\Local\\Continuum\\Anaconda2\\envs\\py36\\lib\\site-packages\\matplotlib\\projections\\__init__.py\u001b[0m in \u001b[0;36mprocess_projection_requirements\u001b[1;34m(figure, *args, **kwargs)\u001b[0m\n\u001b[0;32m     96\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     97\u001b[0m     \u001b[1;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mprojection\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0msix\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstring_types\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mor\u001b[0m \u001b[0mprojection\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 98\u001b[1;33m         \u001b[0mprojection_class\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mget_projection_class\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mprojection\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     99\u001b[0m     \u001b[1;32melif\u001b[0m \u001b[0mhasattr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mprojection\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'_as_mpl_axes'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    100\u001b[0m         \u001b[0mprojection_class\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mextra_kwargs\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mprojection\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_as_mpl_axes\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mC:\\Users\\mgalarnyk\\AppData\\Local\\Continuum\\Anaconda2\\envs\\py36\\lib\\site-packages\\matplotlib\\projections\\__init__.py\u001b[0m in \u001b[0;36mget_projection_class\u001b[1;34m(projection)\u001b[0m\n\u001b[0;32m     65\u001b[0m         \u001b[1;32mreturn\u001b[0m \u001b[0mprojection_registry\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget_projection_class\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mprojection\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     66\u001b[0m     \u001b[1;32mexcept\u001b[0m \u001b[0mKeyError\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 67\u001b[1;33m         \u001b[1;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"Unknown projection '%s'\"\u001b[0m \u001b[1;33m%\u001b[0m \u001b[0mprojection\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     68\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     69\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mValueError\u001b[0m: Unknown projection '3d'"
     ]
    }
   ],
   "source": [
    "# Surface Plot\n",
    "fig = plt.figure()\n",
    "\n",
    "ax = fig.gca(projection='3d')\n",
    "\n",
    "theta0_vals, theta1_vals = np.meshgrid(theta0_vals, theta1_vals) # necessary for 3D graph\n",
    "\n",
    "surf = ax.plot_surface(theta0_vals, theta1_vals, J_vals, cmap=cm.coolwarm, rstride=2, cstride=2)\n",
    "\n",
    "fig.colorbar(surf)\n",
    "\n",
    "plt.xlabel('theta_0')\n",
    "\n",
    "plt.ylabel('theta_1')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# # Contour plot\n",
    "\n",
    "fig = plt.figure()\n",
    "\n",
    "ax = fig.add_subplot(111)\n",
    "\n",
    "# # Plot J_vals as 20 contours spaced logarithmically between 0.01 and 100\n",
    "\n",
    "cset = plt.contour(theta0_vals, theta1_vals, J_vals, np.logspace(-2, 3, 20), cmap=cm.coolwarm)\n",
    "\n",
    "fig.colorbar(cset)\n",
    "\n",
    "plt.xlabel('theta_0')\n",
    "\n",
    "plt.ylabel('theta_1')\n",
    "\n",
    "plt.hold(True)\n",
    "\n",
    "plt.plot(theta[0,0], theta[1,0], 'rx', markersize=10, linewidth=2)\n",
    "\n",
    "plt.show(block=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The purpose of these graphs is to show you that how J(θ) varies with changes in θ<sub>0</sub> and θ<sub>1</sub>. The cost function J(θ) is bowl-shaped and has a global mininum. (This is easier to see in the contour plot than in the 3D surface plot). This minimum is the optimal point for θ<sub>0</sub> and θ<sub>1</sub>, and each step of gradient descent moves closer to this point.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda env:py36]",
   "language": "python",
   "name": "conda-env-py36-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
